fix: use max_output_tokens when available in LiteLLM fetcher #8455
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Prefer max_output_tokens over max_tokens for maxTokens field - Fixes issue where Claude Sonnet 4.5 via Google Vertex was using incorrect token limit - Added comprehensive test coverage for the new behavior Fixes #8454
hannesrudolph
added
the
Issue/PR - Triage
![roomote[bot]](https://avatars.githubusercontent.com/in/1546624?s=60&v=4){kind=small}
| expect(result["bedrock-claude"].supportsComputerUse).toBe(true) | ||
| }) | ||
|
|
||
| it("prefers max_output_tokens over max_tokens when both are present", async () => { |
Contributor
Author
There was a problem hiding this comment.
[P3] Tests: Missing default-fallback case. The PR description mentions covering 'neither field present', but there isn't a test asserting maxTokens defaults to 8192 when both max_output_tokens and max_tokens are absent. Adding one would lock in the intended behavior and prevent regressions.
dosubot
bot
added
size:M
daniel-lxs
approved these changes
Oct 27, 2025
hannesrudolph
added
PR - Needs Review
and removed
Issue/PR - Triage
mrubens
approved these changes
Oct 27, 2025
github-project-automation
bot
moved this from PR [Needs Review]
to Done
in Roo Code Roadmap
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR fixes an issue where LiteLLM was incorrectly using
max_tokensinstead ofmax_output_tokensfor themaxTokensfield, causing errors with Claude Sonnet 4.5 via Google Vertex.Problem
When using Claude Sonnet 4.5 via Google Vertex through LiteLLM, requests were failing with:
The issue was that the code was using
max_tokens(which can be 200k for context) instead ofmax_output_tokens(which is limited to 64k for output).Solution
Modified the LiteLLM fetcher to prefer
max_output_tokenswhen available, falling back tomax_tokensfor backward compatibility:maxTokens: modelInfo.max_tokens || 8192maxTokens: modelInfo.max_output_tokens || modelInfo.max_tokens || 8192Testing
Added comprehensive test coverage to verify:
max_output_tokensis preferred when both fields are presentmax_tokenswhenmax_output_tokensis not availableAll existing tests continue to pass.
Fixes #8454
Important
Fixes LiteLLM fetcher to prefer
max_output_tokensovermax_tokens, resolving token limit errors with Claude Sonnet 4.5.max_output_tokensinstead ofmax_tokensformaxTokensfield inlitellm.ts.max_tokensifmax_output_tokensis unavailable.litellm.spec.tsto verify preference formax_output_tokensand fallback behavior.This description was created by
for a113acc. You can customize this summary. It will automatically update as commits are pushed.